MacFormat 1995 March

home *** CD-ROM | disk | FTP | other *** search

/ MacFormat 1995 March / macformat-022.iso / Shareware City / Utilities / MacDOS™ 2.0.0ƒ / User's Guide / B < prev next >

Wrap

Text File | 1994-11-23 | 29.5 KB | 601 lines | [TEXT/ttxt]

B Extension Programming Introduction With release 2.0.0, MacDOS™ supports a new and powerful piping mechanism. This allows programmers to extend the functionality of MacDOS through independent filter applications. The purpose of this section is to explain how the mechanism works and how programmers can make the best use of it. To provide more than a simple cookbook, we have structured this section as follows: Functional Specification (what pipes are and what we expect from them); Design (how we implement pipes on the Mac); Programmer's Guide (the pipe toolbox and how to use it). Functional Specification In an operating system based on a command line interface (CLI), each command is usually associated with a process. A pipe is a means of sending information from one process to another. Like water in a physical pipe, information in a software pipe only flows in one direction and what enters a pipe comes out of the other end in the same order (FIFO: First In First Out). The purpose of such a mechanism is to feed the second process with the output of the first one. Predictably, another pipe can transfer information from the second process to a third one, still another pipe from the third to the fourth, etc. The end result is a chain of processes in which each process acts as a filter. Only the two processes at the beginning and at the end of the chain interact with the user (typically, via the keyboard and the monitor). In principle, each process keeps checking whether the incoming pipe has something ready to be processed, does its processing on the bits and pieces it gets, and sends the result immediately to the outgoing pipe. In practice, it is the operating system that takes care of the piping. Each process still believes that it is getting its input from the keyboard and sending its output to the monitor. As the processes are unaware that they communicate via pipes, all input/output is of ASCII characters rather than binary data. The Mac OS is not built around a CLI and does not directly support I/O redirection and piping and the concept of standard I/O devices is missing. Although MacDOS provides a CLI for the Mac, it operates at the application level. Therefore, its console window is not directly accessible to other applications. As a consequence, Mac applications (unlike UNIX® programs) must be aware of piping. Furthermore, as the piping mechanism is outside the OS, it is bound to be less efficient. In view of all these considerations, we can specify piping for the Mac as follows: • Piping shall be capable of transferring several characters in a single operation. This will reduce the overhead due to context switching between processes and to message handling within the processes themselves. • The last filter of a chain shall send its output back to MacDOS, so that it can be displayed in the console window or redirected to a disk file. • Any filter of a chain shall be able to report errors back to MacDOS, so that they can be displayed in the console window. • The failure of any filter of a chain shall cause the quitting of all other filters and be detected by MacDOS so that it can be reported to the user. • The user shall be able to terminate the operation of a chain of filters by typing a cntl-C/cmd-dot as with any other MacDOS command. • MacDOS shall be able to pass to any filter its specific switches as typed by the user in the command line. Additionally: • The user shall be able to obtain on-line help by typing "HELP filterName" at the MacDOS prompt. • MacDOS and the filters shall reside on the same Mac. That is, it shall not be possible to start MacDOS on system A and pipe a command to a filter which is on system B. Design This section describes how the generic functional requirements listed above translate into practice. That is, how MacDOS and filters exchange information. General Strategy Under the Mac OS, there are two basic ways for processes to exchange information: Apple Events (AEs) and Process to Process Communication (PPC). Communication via AEs is connectionless, in the sense that each AE contains information about its destination process (a bit like sending letters through the mail). Also, when you send an AE, you have plenty of options on how you want the destination process to respond to it. This provides a lot of flexibility but introduces quite a bit of overhead as well. Communication via the PPC toolbox is connection oriented, because you have to establish a communication session with the destination process before you can begin to send data to it, but then you can easily exchange messages without addressing them (a bit like making a telephone call). PPC requires some additional overhead at the beginning and limits flexibility later on, but it reduces the system overhead to do the actual data transfer. In view of these considerations, MacDOS uses PPC to implement the pipes. Before two processes can exchange information via PPC, they have to take the following steps: • Each process opens a communication port (say, process 1 opens port A and process 2 opens port B). • One of the processes (say, process 1), tells the OS that it is ready to accept PPC sessions. • The other process (in this example, process 2) tells the OS that it would like to start a session with port A. Normally, PPC operations are conducted asynchronously. That is, you start an operation by executing a function which returns immediately. To know when the OS has completed the operation you have requested, you have two possibilities: either you provide the OS with a call-back function (which the OS executes upon completion of the operation), or you check a flag. MacDOS checks the flag, mainly because it is a simpler (ie. safer) mechanism. As PPC sessions are bidirectional, filters can easily report back to MacDOS local errors and errors detected when communicating with the next filter. Data Structures Each PPC message contains two fields which can be used to store information: • A long integer (4 bytes) called "userData". • A buffer called "dataBuffer". MacDOS uses the four bytes of userData as follows (in order of increasing memory address): type specifies the content of the userData field. seqis a sequence number set by MacDOS to identify each message. srcstands for source and identifies the process which generates the message. dststands for destination and identifies the process which should operate on the message. dataBuffer always contains a P-string (ie. an unsigned character array with its first byte set to the number of characters that form the body of the message). The possible message types and what they mean are listed in the table at the top of the next page. Procedures A filter needs to: • Initialise the piping mechanism (open the PPC port, establish a session with the preceding filter, and initialise state variables). • Process every incoming message. • Forward to the next process every data message filtered. • Report errors to MacDOS. • Clean up (most importantly, close the PPC port, that otherwise remains open even after the filter has quit). Type Corresponding content of dataBuffer pipeDataMess To be filtered. This is what we use pipes for. pipeParmMess Filter configuration parameters (ie. switches). pipeNextMess MacDOS uses this message to direct a filter to establish a session with the PPC port of the next filter of the chain. It consists of the port name followed by the ID that MacDOS has assigned to the next filter. pipeErrMess The first byte is the Most Significat Byte of an error code of type OSType. The second byte of the dataBuffer is the Least Significant Byte of the same error code. When the error code is pipeFilteringErr, the following bytes of the message (if any) are characters used by the source filter to send to MacDOS an error message in clear. MacDOS establishes the chain of filters as follows: 1 It opens its own PPC port. 2 It launches the first filter application. 3 It establishes a PPC session with the resulting process (for this to be possible, each filter must open a PPC port named like its own application file). When establishing the session, the userData field contains the ID of the filter (typecast from char to long). 4 It configures the filter by sending its paramaters as specified in the command line (pipeParmMess). 5 It launches the second filter application. 6 It directs the first filter to start a PPC session with the resulting new process (via a pipeNextMess to the first filter). 7 It configures the second filter via a pipeParmMess like it was done for the first filter. 8 It repeats steps 5 to 7 for all following filters in the chain until the last filter is configured. 9 It directs the last filter to start a PPC session with MacDOS. 10 It accepts the session opened by the last filter. At this point, the full chain is completed and the actual processing of data can begin (via messages of type pipeDataMess). The message types mentioned so far only travel downstream (ie. from left to right in the chain of filters as they appear in the command line), while error messages only travel upstream (ie. back to MacDOS through all intermediate filters in the reverse order). Each filter is connected to two sessions. The session with the preceding process (upstream) implements the incoming pipe, while the session with the following process (downstream) implements the outgoing pipe. A filter only analyses and acts on messages which have as destination the ID of the filter. All other messages are forwarded to the other session unchanged. This is how MacDOS can build the chain one filter at a time and how filters can report errors back to MacDOS. If you are familiar with concepts of Data Communication, the PPC represents a data-link protocol layer (OSI level 2), while the way in which MacDOS uses the userData field of the messages implements a network protocol layer (OSI level 3). As it will become clear in the Programmer's Guide section, all configuration and forwarding operations can be "hidden" inside a single polling function, so that the programmer of a new filter does not need to be concerned with them. The basic structure of a filter is shown in Fig B1. In most cases, MacDOS generates a data message every time a CR-terminated line of text is completed. When a filter receives a data message, it processes the message, formats the result of the processing into another pipeDataMess, and sends it to the next filter. When MacDOS receives a message from the last filter, it displays or stores it into a file as requested by the user in the command line. The source-destination pair of data messages always identifies adjacent processes. That is, data messages are always processed after crossing a single pipe. To avoid that MacDOS remains "hanging" (until it times out), filters must always forward a data message to the next filter. If, as a result of filtering, a filter has nothing to forward, it must send a message with a zero- length string in its dataBuffer. On the other hand, nothing prevents filters from sending several data messages to the next process as a reaction to a single incoming data message. It is particularly important that filters send an empty data message to the outgoing pipe when they receive one from the incoming pipe. The reason is that MacDOS sends an empty message to flush the filter chain before prompting the user for a new command. When a filter is waiting for incoming messages, it normally checks both the incoming and outgoig pipes. Nevertheless, after sending an error message to MacDOS, it only checks the incoming pipe and quits when it discovers that the session has been closed. The Polling Startegy in Detail The two PPC functions used to communicate through PPC sessions are PPCRead and PPCWrite. The filter shell included in the MacDOS release uses both functions asynchronously. DO NOT MODIFY ANY VARIABLE USED IN AN ASYNCHRONOUS PPC-CALL BEFORE THE OPERATION IS COMPLETED. If you do, you will cause unpredictable system errors and will have to restart your Mac. A filter keeps an outstanding asynchronous PPCRead for each started session. This means that as soon as it receives a message from a pipe, it saves the message and immediately initiates a new Read operation. When checking whether a new message has arrived (see the PipePollOnce function described in the next section), a filter always checks the outgoing pipe first. This gives priority to error messages (which travel upstream). The filter then keeps checking each pipe in a round robin fashion. After checking a pipe, the filter executes WaitNextEvent so that the Mac OS can activate other processes. To send a message to a pipe (data messages to the outgoing pipe or error messages to the incoming pipe) a filter executes an asynchronous PPCWrite. It then only checks for the completion of the PPCWrite before sending the next message to the same pipe. Typically, a filter spends most of its time polling the pipes and processing data messages. As explained above, the pipe polling generates a lot of calls to WaitNextEvent. If you type a cntl-C (or a cmd-dot), Wait NextEvent returns successfully and the filter sends to MacDOS the error code userCanceled. In fact, keyboard events are the only events that a filter handles. Handling of the Sequence Number MacDOS uses the seq field of the userData to decide whether it can send a new message and whether it can prompt the user for the next command. To achieve this result, MacDOS stamps all outgoing messages with ever increasing sequence numbers. As sequence numbers are stored in 8 bits, they wrap around 256 (ie. they restart from 0 after reaching 255). The Mac OS ensures that messages are delivered in the same order in which they are sent, but MacDOS checks the sequence of incoming messages and aborts the command when it receives a message with a sequence number lower than that of the previous message. This is a safety precaution to protect MacDOS against some effects of badly designed filter applications (they could corrupt the userData of messages). When sending a data message to the outgoing pipe, a filter uses the sequence number of the last message received from the incoming pipe. Therefore, if a filter responds to a single incoming message by sending several messages to the next filter, all those messages have the same sequence number. Before sending a data message, MacDOS always updates the sequence number. Then, after sending the message, MacDOS expects to receive from the incoming pipe the filtered data message with the same sequence number. Only after receiving such a message, can MacDOS resume execution of the original command (this is why filters must always forward something). Now, when a filter sends several messages with the same sequence number, MacDOS does not realise it until it begins waiting for the reply to its next message. In that case, MacDOS keeps processing all the incoming messages until it receives a message with the expected sequence number. In this way, unless a filter "misbehaves", MacDOS avoids the piling up of incoming messages. After completing execution of the original command, MacDOS sends a data message with an empty dataBuffer. This effectively "flushes" the chain of pipes and ensures that no filter-generated messages are outstanding. Only after receiving the corresponding empty message from the incoming pipe, does MacDOS close the PPC port and prompts the user for a new command. Data Flow This section illustrates in a graphic form how messages travel through the chain of filters. For this purpose, a chain of two filters is used. In all the following diagrams, messages travelling downstream are represented by arrows pointing to the right. Fig B2 shows how MacDOS sets the chain up. The dashed lines correspond to interactions with the PPC other than for sending messages. Each number IDentifies the destination filter, both for messages and for other actions. Note that each filter performs a PipeInit (which includes accepting incoming PPC session requests) immediately after launch, while MacDOS only accepts sessions after directing the last filter to start one. Once the initialisation is completed, both filters are in their default state, waiting for data messages. Fig B3 shows how the normal data transfer looks in a simple case. After sending a data message, MacDOS waits for the corresponding data message from its incoming pipe. Note that both filters are in their default state before receiving the data message and after completing its processing: they poll the pipes waiting for something to do. Fig B4 provides an example of data transfer in which a filter sends two messages for each message it receives. Note how MacDOS polls again after displaying 1b. It does so because MacDOS has already sent a message with sequence number 2, while 1b has still sequence number 1. MacDOS will only see 2b while waiting for 3, etc. After sending the last line with, say, sequence number N, MacDOS will send an empty message with sequence number N + 1, so as to detect possible Nb, Nc, etc. Fig B5 shows how filters report error messages back to MacDOS. The filters quit when they find out that their incoming pipe has disappeared (ie. that the incoming session is no longer there). Therefore, as soon as MacDOS closes its session with the first filter, that filter quits. This causes the disappearence of the next pipe, etc. This cascading effect terminates when the last filter quits. Programmer's Guide This section describes the functions provided in the "filter shell" project. It also tells you how to structure and build a filter application. Pipe Toolbox The pipe.c/pipe.h sources provide the following functions (in alphabetical order): void PipeInit(pipeParmsFun_t *parmsFun); PipeInit prepares the filter to handle pipe messages. It opens the PPC port and only returns after establishing a session with the process upstream. parmsFun is a pointer to a function which accepts configuration parameters as they appear in the command line and saves them in variables accessible during filtering. Set it to nil if your filter is not configurable. void PipePoll(unsigned char *inMess); PipePoll waits for messages from either PPC session. It automatically handles all messages unless they are data messages received from the incoming pipe and directed to the filter itself. inMess is the pointer to the buffer where PipePoll can store an incoming message. inMess should be at least pipeSize byte long and will contain a P- string when PipePoll returns successfully. void PipeReportError( OSErr err, unsigned char *errInClear ); PipeReportError sends an error message to MacDOS. It can be used to report errors encountered during filtering. err is the error code. To have MacDOS display a proprietary error string, set err to pipeFilteringErr. MacDOS will then display errInClear. errInClear is a proprietary error string that you would like MacDOS to display in the console window. MacDOS only looks at this string if you set err to pipeFilteringErr. If you use PipeReportError to report a Mac system error rather than a proprietary error, you can set errInClear to nil. void PipeSendData(unsigned char *outMess); PipeSendData sends a message to the next filter downstream. outMess is the pointer to the buffer containing the message to be sent. outMess must contain a P-string and cannot occupy more than pipeSize bytes. Note that the Pipe functions only return if they are successful. Otherwise, they automatically attempt to report to MacDOS the first error they encounter. They then wait for MacDOS to "cut" the pipes. NEVER "DROP OFF THE BOTTOM" OF A FILTER. Let the Pipe functions take care of quitting the filter application. This will ensure that the filter closes its PPC port before quitting. If the PPC port remains open, you will not be able to re-use the same filter unless you restart your Mac (or change the filter application name, although this technique would lead to a proliferation of "ghost" ports. Only do it in emergency). Filter Structure The "filter shell" hides most of the complexities of a filter to the programmer. The resulting simplified structure of a typical filter is shown in Fig B6. Debugging Filter Projects In order to debug filters, you have to know how to do two things: 1 Launch your filter project instead of a compiled filter application. 2 Have MacDOS wait for you when you pause execution of the filter project with a breakpoint. The first issue exists because MacDOS rightly recognises that your filter project is not a filter application. It expects a file of type 'APPL' and creator 'mFLR' but it finds a file of type 'PROJ' and creator 'KAHL' (only if you are using THINK C, but the discussion remains valid for other compilers). To trick MacDOS, do as follows: • Place an alias of the file fakeFilter.π in the folder where you keep MacDOS. • Rename the alias like your filter project. • Run your filter project, set the breakpoint[s], and let it go. • Click in the MacDOS console window to push the filter process to the background and bring MacDOS to the front. • Type the command that you want to use to test your filter, but only type the name of your filter project, without its real path. MacDOS will see the renamed fakeFilter and will happily launch it (it will die at once). MacDOS will then succeed in opening a PPC session with the filter port, because your project was already running and had already opened the port correctly. This trick works because the name of a filter application coincides with the name of the PPC port opened by the filter. The second issue exists because MacDOS normally waits a couple of seconds for data messages to be looped back through the filter chain. If you pause execution of the filter with a breakpoint, MacDOS will timeout and abort the command. The solution is simple because MacDOS sets the timeout to the number of seconds specified in the global variable TIMEOUT. Therefore, if you type "set timeout=3600" at the prompt, MacDOS will wait for one hour before timing out. Building a Filter Application The simplest way to build a filter is to duplicate and rename the files filter.c, filter.π, and filter.π.rsrc . Within filter.c, you only need to modify the content of the function doFiltering (and HandleParameters if you want to use switches in the command line). The file type of filters is 'APPL' (no 'APPE', please), and their creator is 'mFLR'. The fact that all filter applications have the same creator should not pose any problem, because MacDOS looks for them by name, and no document file type is defined for the same creator. Therefore, as you will not be able to start a filter by double clicking on a document, the Mac OS will never be placed in the position of having to choose a filter application on the basis of its creator string. The project filters.π gives you some examples of how to define filter parameters and forward multiple messages to MacDOS. The HELP command of MacDOS now supports on-line help of filter applications. What you need to do in order to set up an help message for your filter is to include a resource of type 'TEXT' named "help" in its resource fork (the name "help" is not case sensitive). If you open filter.π.rsrc with resEdit, you will see that it already contains the default help message: Sorry, no help available for this filter. Abbreviations AE Apple Events ASCII American Standard Code of Information Interchange. This is the encoding of characters chosen by IBM, Apple, and most other computer manufacturers. The standard encoding assigns a value between 0 and 127 to 128 characters. Apple defines 128 additional characters to cover all 256 possible combinations of 8 bits. CLI Command Line Interface. CR The ASCII character Carriage Return. This is how the Mac OS separates lines of text. DOS Disk OS (most common Operating System for PCs). FIFO First In First Out IBM Industrial Business Machines (although they have recently changed the meaning of 'IBM') ISO International Organisation for Standardisation OS Operating System OSI Open System Interconnection. ISO's layered model for communication protocols. PPC Process to Process Communication toolbox RH Rainbow Hill Pty Ltd.